Skip to content

fix(server): honor configured output default across APIs - #411

Merged
jason-fxz merged 1 commit into
FlashML-org:mainfrom
earlvanze:fix/max-output-tokens-all-apis
Sep 12, 2026
Merged

fix(server): honor configured output default across APIs#411
jason-fxz merged 1 commit into
FlashML-org:mainfrom
earlvanze:fix/max-output-tokens-all-apis

Conversation

@earlvanze

Copy link
Copy Markdown
Contributor

Closes #395.

Threads the server-level --max-output-tokens fallback through the Chat Completions and Completions adapters, as the Responses adapter already does. Explicit request limits still take precedence; absent server configuration retains the 32k fallback.

Validation:

  • python3 -m py_compile for changed modules
  • git diff --check
  • Focused pytest suite is not runnable in this checkout because pytest/runtime dependencies are not installed.

@jason-fxz

Copy link
Copy Markdown
Collaborator

Tested end-to-end on top of current main: with --max-output-tokens 64, /v1/chat/completions and /v1/completions without max_tokens now stop at the configured limit (63 tokens vs 2032 on main, clamped from the 32k constant); explicit limits unchanged, server tests pass. LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] --max-output-tokens is ignored by /v1/chat/completions and /v1/messages (only /v1/responses honours it)

2 participants